Image editing apparatus, image editing method and program

ABSTRACT

Method and information processing apparatus for generating an edited work including a subset of a plurality of scenes in an image material. The information processing apparatus includes a memory configured to store the image material including the plurality of scenes. The information processing apparatus further includes a processor configured to select, for an n-th scene of the edited work, a plurality of first candidate scenes from the plurality of scenes based on at least one feature of a preceding (n−1-th) scene of the edited work and features of the plurality of scenes. The processor is also configured to generate a graphical user interface including a scene selection area and a preview area. The scene selection area includes one or more first candidate images corresponding to at least one of the plurality of first candidate scenes, and the preview area includes a preview of the preceding (n−1-th) scene.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.12/838,997, filed Jul. 19, 2010, which contains subject matter relatedto that disclosed in Japanese Priority Patent Application JP2009-176572, filed in the Japan Patent Office on Jul. 29, 2009, theentire content of both of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image editing apparatus, an imageediting method, and a program.

2. Description of the Related Art

A moving image captured and recorded by an imaging apparatus such as acamcorder includes a scene where an image was not successfully capturedor a scene that is unnecessary, and thus the moving image has to beedited to make it look smooth-running. For this purpose, various editingapplication software for editing a moving image in an editing apparatussuch as a personal computer (PC) or the like have been developed.

However, for a user who is not used to editing a moving image, there areobstacles relating to an editing environment and an editing skill, andthus, the editing application mentioned above can hardly be said to bewidely used. For example, the obstacle relating to the editingenvironment may be that selection of a PC and an editing applicationsuitable for the PC is difficult, that a PC has to be of a highspecification, or that learning of application operation is bothersome.Also, the obstacle relating to the editing skill may be that a task ofselecting a position at which a scene in a moving image is to bedivided, of selecting or deleting a scene, of connecting a plurality ofscenes, or of specifying an image switching effect to be applied betweenscenes is difficult.

Therefore, various PC applications have been developed for automaticediting of a moving image. Among these, according to a fully automaticediting application, a user can achieve an edit result of a moving imageby simply specifying a moving image to be edited and a desired template.For example, JPA-2004-159331 discloses an automatic editing applicationthat calculates the importance of a sub-shot that is extracted from avideo and generates a video summary according to the importance.

However, according to the fully automatic editing application, a user isnot allowed to adjust a moving image that has been automaticallyadjusted, and a desired piece of work might not be obtained.Accordingly, an application that allows adjustment to be made to amoving image that has been automatically edited is also developed. Forexample, “VAIO de kantan! Suteki na movie wo tsukuttemiyo!,” [online],Sony Corporation, [searched on Jul. 16, 2009], Internet<URL:http://vcl/vaio.sony.co.jp/support/special/appl/dvd_howto05-4.html>discloses an automatic editing application capable of adding a new sceneto or deleting a scene from a moving image that has been automaticallyedited by an application or of rearranging the order of scenes by a useroperation.

SUMMARY

However, according to the editing application described above whichallows adjustment after the automatic editing, a user has to perform,after the automatic editing, user operation of checking the necessity ofmultiple scenes and switching or adding each scene, and thus there isstill a room for improvement of the operating system. Also, it is notable to present a user with a plurality of recommended scenes that maybe suitably linked after a certain scene.

The present invention addresses the above-identified issues, and enablesa user to be presented with a plurality of candidate scenes that may besuitably linked after a certain scene at the time of adjustment of anautomatically edited image work, and also enables the user to easilyproduce an image work that is to the user's liking by a simple operatingsystem.

According to an exemplary embodiment, an information processingapparatus is provided for generating an edited work including a subsetof a plurality of scenes in an image material. The informationprocessing apparatus includes a memory and a processor. The memory isconfigured to store the image material including the plurality ofscenes. The processor is configured to select, for an n-th scene of theedited work, a plurality of first candidate scenes from the plurality ofscenes based on at least one feature of a preceding (n−1-th) scene ofthe edited work and features of the plurality of scenes, and to generatea graphical user interface including a scene selection area, whichincludes one or more first candidate images corresponding to at leastone of the plurality of first candidate scenes, and a preview area,which includes a preview of the preceding (n−1-th) scene.

Further, according to another exemplary embodiment, a method is providedfor generating an edited work including a subset of a plurality ofscenes in an image material. The method includes storing the imagematerial including the plurality of scenes. A plurality of firstcandidate scenes from the plurality of scenes are selected by aprocessor of the information processing apparatus, for an n-th scene ofthe edited work, based on at least one feature of a preceding (n−1-th)scene of the edited work and features of the plurality of scenes.Further, a graphical user interface is generated by the processor of theinformation processing apparatus. The graphical user interface includesa scene selection area, which includes one or more first candidateimages corresponding to at least one of the plurality of first candidatescenes, and a preview area, which includes a preview of the preceding(n−1-th) scene.

Further, according to another exemplary embodiment, a non-transitorycomputer-readable storage medium is provided for generating an editedwork including a subset of a plurality of scenes in an image material.The non-transitory computer-readable storage medium stores instructions,which when executed by a processor, causes the processor to store theimage material including the plurality of scenes. A plurality of firstcandidate scenes from the plurality of scenes are selected by aprocessor of the information processing apparatus, for an n-th scene ofthe edited work, based on at least one feature of a preceding (n−1-th)scene of the edited work and features of the plurality of scenes.Further, a graphical user interface is generated by the processor of theinformation processing apparatus. The graphical user interface includesa scene selection area, which includes one or more first candidateimages corresponding to at least one of the plurality of first candidatescenes, and a preview area, which includes a preview of the preceding(n−1-th) scene.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an example of display of an edit screenaccording to an exemplary embodiment of the present invention;

FIG. 2 is a block diagram showing a hardware configuration of an imageediting apparatus according to an exemplary embodiment;

FIG. 3 is a block diagram showing a functional configuration of theimage editing apparatus according to an exemplary embodiment;

FIG. 4 is a diagram showing examples of features of an image materialaccording to an exemplary embodiment;

FIG. 5 is a flow chart showing an image editing method that uses animage editing apparatus according to an exemplary embodiment;

FIG. 6 is a conceptual diagram showing an example of an analysis processof an edit material group according to an exemplary embodiment;

FIG. 7 is a conceptual diagram showing an example of a selection processfor a candidate scene according to an exemplary embodiment;

FIG. 8 is a diagram showing an example of display of an edit screen inaccordance with a story-based edit mode according to an exemplaryembodiment;

FIG. 9 is a diagram showing an example of display of an edit screen inaccordance with a music-driven edit mode according to an exemplaryembodiment;

FIG. 10 is a diagram showing an example of display of an edit screen inaccordance with scene drop edit mode according to an exemplaryembodiment;

FIG. 11 is a diagram showing an example of display of an edit screendisplaying a selection menu according to an exemplary embodiment;

FIG. 12 is a diagram showing an example of display of an edit screendisplaying a caption according to an exemplary embodiment; and

FIG. 13 is a diagram showing image editing apparatuses according tomodified examples of the present invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT(S)

Hereinafter, preferred exemplary embodiments of the present inventionwill be described in detail with reference to the appended drawings.Note that, in this specification and the appended drawings, structuralelements that have substantially the same function and structure aredenoted with the same reference numerals, and repeated explanation ofthese structural elements is omitted.

Additionally, description will be made in the order shown below.

1. Outline of Editing Method

2. Example of Display of Edit Screen

3. Hardware Configuration of Image Editing Apparatus

4. Functional Configuration of Image Editing Apparatus

5. Image Editing Method

6. Application Example

7. Summary

<1. Outline of Editing Method>

First, an outline of an image editing method of an image editingapparatus according to the first exemplary embodiment of the presentexemplary embodiment will be described.

The image editing apparatus according to the present exemplaryembodiment includes an automatic edit function of automatically editingan image material based on a feature (attribute) of the image material,and a manual adjustment function of adjusting, according to a userinput, an image work that has been produced by the automatic editing tocomplete the edited work.

The image material here is image data formed from moving images or stillimages. For example, the image material is image data that has beencaptured and recorded by an imaging apparatus, such as a digitalcamcorder, a digital still camera or the like, but it may also be imagedata created by image processing, such as CG (computer graphics), anillustration or the like. Furthermore, the edited work is the result ofediting the image material, i.e. image data that is obtained by editingone or more image materials. Additionally, in the present specification,an image includes a moving image and a still image. Also, a scene is apart or all of the sections of a moving image. The scene is a unit forediting to be used at the time of editing a moving image.

With the automatic editing function, the image editing apparatusanalyses an image material and obtains the feature of the imagematerial, and based on the feature, divides a moving image in the imagematerial into a plurality of scenes. Then, based on the features ofthese plurality of scenes, the image editing apparatus extractsappropriate scenes from the plurality of scenes, arranges theseextracted scenes in an appropriate order, and automatically produces anedited work. Such an automatic editing function enables the automaticcreation of an appropriate edited work according to the feature of theimage material, and thus convenience of a user can be achieved.

With the manual adjustment function, the image editing apparatusdisplays an edit screen for the edited work created by the automaticediting function, and adds or deletes a scene to/from the edited workthat has been automatically edited or rearranges the order of thescenes. Such manual adjustment function enables a user to easily modifythe edited work that has been automatically edited to an edited work (anedit result) that is to the user's liking, and thus the quality of theedited work can be improved.

As described above, the image editing apparatus according to the presentexemplary embodiment uses the automatic editing function and the manualadjustment function to edit an image material and produce an edited workdesired by a user. At the time of manual adjustment, the image editingapparatus presents on an edit screen a plurality of candidates(candidate scenes) for scenes that may be suitably linked after acertain scene. Accordingly, a user can produce an edited work that isdesirable to the user by sequentially selecting from the beginningscenes that are to the user's liking among the plurality of candidatescenes presented on the edit screen. As described, the image editingapparatus according to the present exemplary embodiment has itscharacteristic in providing a user interface (edit screen) having anextremely simple operation and which enables a user to easily produce anedited work that is to the user's liking.

<2. Example of Display of Edit Screen>

Next, the outline of an edit screen 1 of the image editing apparatusaccording to the present exemplary embodiment will be described withreference to FIG. 1. FIG. 1 is a diagram showing an example of displayof the edit screen 1 of the image editing apparatus according to thepresent exemplary embodiment.

As shown in FIG. 1, the edit screen 1 is a GUI (Graphical UserInterface) screen for a user to manually adjust an edited work that hasbeen automatically edited by the image editing apparatus. The editscreen 1 is displayed on a display device (not shown) of the imageediting apparatus by application software that is installed in the imageediting apparatus. A user can check the outline of the edited work thathas been automatically edited by looking at the edit screen 1, and also,can perform manual editing such as addition/deletion/rearrangement of aplurality scenes making up the edited work.

As described above, based on the feature of one or more image materials,the image editing apparatus automatically edits the one or more imagematerials into an edited work by dividing a moving image in the one ormore image materials into a plurality of scenes and combiningappropriate scenes among the plurality of scenes in an appropriateorder. The edited work that has been automatically edited in this manneris made up of a combination of images of a plurality of scenes extractedfrom the one or more image materials. By performing specific operationwhile looking at the edit screen 1, a user can adjust (reedit) theedited work that has been automatically edited to a work that is to theuser's liking by deleting an unnecessary scene in the edited work, byadding a new scene thereto, or by changing the order of the scenes. Theedit screen 1 is a GUI that enables a user to perform such manualediting by a simple operation.

In the following, the configuration of the edit screen 1 will bedescribed in detail. Additionally, in the following, a case will bedescribed where an edited work that has been generated by the automaticediting is made up of m scenes (m is any integer that is 2 or more) andwhere a user selects, on the edit screen 1, m scenes that make up theedited work, one by one from the beginning. The edited work is a finalresult of editing obtained by automatically and manually editing animage material.

As shown in FIG. 1, the edit screen 1 includes, on one screen, a sceneselection area 2, a next-scene display area 3, a preview area 4, and aroll film area 5. Each area on the edit screen 1 will be described indetail below.

First, the scene selection area 2 will be described. The scene selectionarea 2 is an area that displays a plurality of candidate scenes 20A to20C, which are the candidates for a scene n-th from the beginning of theedited work, and that are used by a user to manually select onecandidate scene from the plurality of candidate scenes 20A to 20C. Inthe example shown, the scene selection area 2 is arranged approximatelyin the middle of the edit screen 1, and images of three candidate scenes20A, 20B and 20C are arranged in a vertical direction in the sceneselection area 2. Additionally, the number of candidate scenes to bepresented in the scene selection area 2 is not limited to three as inFIG. 1, and any arbitrary number (for example, two or four or more) ofcandidate scenes 20 may be presented. Also, the image of the candidatescene 20 may be a moving image or a still image as long as it is atleast a part of the reproduction image of the candidate scene 20. Forexample, only the candidate scene 20B that is currently selected by acursor 21 may be made to be a moving image, and other candidate scenes20A and 20C may be still images.

The image editing apparatus automatically selects as the candidates foran n-th scene of the edited work a specific number of candidate scenes20 which may be suitably linked after a preceding scene (n−1-th scene),based on the feature of each scene of an image material. At this time,the image editing apparatus obtains the recommendation level for eachcandidate scene 20 for the n-th scene based on the feature of each sceneof the image material and based on the relationship between the featureof each scene and the feature of the n−1-th scene, and automaticallyselects a specific number (for example, three) of candidate scenes 20sequentially from the highest recommendation level. The images of aplurality of candidate scenes 20 that are automatically selected in thismanner are displayed in the scene selection area 2 in accordance withthe order of recommendation level. In the example shown, the candidatescene 20B displayed in the middle in the vertical direction has thehighest recommendation level, and the candidate scene 20A on the top hasthe next highest recommendation level, followed by the candidate scene20C on the bottom.

As described, a plurality of candidate scenes 20 that may be suitablylinked after the preceding scene (n−1-th scene) are presented in thescene selection area 2 on the edit screen 1. This enables a user toeasily select the candidate scene 20B that is to the user's liking asthe n-th scene of the edited work by looking only at a specific numberof candidate scenes 20A to 20C that are recommended.

The cursor (indicator) 21 displayed in the scene selection area 2 is atool for selecting one candidate scene from the candidate scenes 20A to20C. A candidate scene 20B specified by the cursor 21 indicates thecandidate scene that is currently selected in the scene selection area2. A user selects the candidate scene 20B that is to the user's likingfrom the candidate scenes 20A, 20B and 20C presented in the sceneselection area 2 by moving the cursor 21 up and down by pressing an upbutton 22A or a down button 22B.

Additionally, by pressing left/right buttons 23A and 23B of the cursor21, an image switching effect to be applied between the candidate scene20 (n-th scene) that is currently selected and scenes to be connectedbefore and after the candidate scene 20 (n−1-th and n+1-th scenes) canbe set. The details of the setting of this image switching effect willbe described later.

Furthermore, an icon 24 is displayed near each candidate scene 20 in thescene selection area 2 as information indicating the feature of thecandidate scene 20. Here, the information indicating the feature of thecandidate scene 20 is information indicating the reason why thecandidate scene 20 is recommended. A picture according to the feature ofeach candidate scene 20 is shown in the icon 24.

For example, an icon 24A indicates that the candidate scene 20A is ascene with a “large number of people,” and an icon 24B indicates thatthe candidate scene 20B is a scene including a “speaking voice”. Also,icons 24C indicate that the candidate scenes 20B and 20C are scenes with“small number of people,” and an icon 24D indicates that the candidatescene 20B is a scene including a “laughing voice”. Also, an icon 24Eindicates that the candidate scene 20C is a scene including a “blurredimage,” and an icon 24F indicates that the candidate scene 20C is ascene including a “smiling face”.

As described, the icon 24 indicating the feature (recommendation reason)of the candidate scene 20 is displayed in the scene selection area 2 inassociation with the candidate scene 20. This enables a user to easilygrasp the feature and the recommendation reason of a candidate scene 20by referring not only to the image of the candidate scene 20 but also tothe icon 24.

As described above, in the scene selection area 2, a plurality ofcandidate scenes 20A to 20C that are recommended as the n-th scene ofthe edited work are presented, and the candidate scene that is to theuser's liking is selected from the candidate scenes 20A to 20C. As thisselection operation, the user may set the cursor 21 to a desiredcandidate scene 20B and press a determination button not shown, forexample. Additionally, in the example shown, a state is shown where athird scene from the beginning of the edited work is selected (n=3).After the selection of the third scene (for example, the candidate scene20B) is completed, images of the candidates for the fourth scene (forexample, candidate scenes 30D, 30E, and 30F) are displayed in the sceneselection area 2 so as to be selectable.

Next, the next-scene display area 3 will be described. The next-scenedisplay area 3 is an area for displaying, for respective candidatescenes 20 for the n-th scene, a plurality of candidate scenes 30A to 30Ifor the n+1-th scene of the edited work as the candidates for the n+1-thscene from the beginning of the edited work. In the example shown, thenext-scene display area 3 is arranged at the right side of the editscreen 1, and three groups of candidate scenes 30A to 30C, 30D to 30F,and 30G to 30I, nine scenes in total, are arranged in a verticaldirection in the next-scene display area 3. Additionally, the image ofthe candidate scene 30 may be a moving image or a still image as long asit is at least a part of the reproduction image of the candidate scene30. However, when considering the visibility of the edit screen 1, theimage of the candidate scene is preferably a still image.

In the next-scene display area 3, the candidate scenes 30A to 30I aredisplayed each in association with one of the candidate scenes 20A to20C that are for the n-th scenes displayed in the scene selection area2. In the example shown, a plurality (for example, three) of candidatescenes 30A to 30C, 30D to 30F, and 30G to 30I are respectivelyassociated with the candidate scene 20A, 20B, and 20C. The candidatescenes 30A to 30C are candidates for a scene that may be suitably linkedafter the candidate scene 20A, the candidate scenes 30D to 30F arecandidates for a scene that may be suitably linked after the candidatescene 20B, and the candidate scenes 30G to 30I are candidates for ascene that may be suitably linked after the candidate scene 20C.Additionally, the number of the candidate scenes 30 to be presented inthe next-scene display area 3 for each candidate scene 20 is not limitedto three as in FIG. 1, and an arbitrary number (for example, one, two,or four or more) of candidate scenes 30 may be presented.

The image editing apparatus automatically selects, as the candidate forthe n+1-th scene of the edited work, the candidate scene 30 (n+1-thscene) that may be suitably linked after the preceding scene (n-thscene) for each of the candidate scenes 20A, 20B and 20C for the n-thscenes, based on the feature of the image material. At this time, as inthe case of the candidate scenes 20 for the n-th scene described above,the image editing apparatus obtains the recommendation level for eachcandidate scene 30 for the n+1-th scene based on the feature of eachscene of the image material and based on the relationship between thefeature and the feature of the candidate scene 20 for the n-th scene,and automatically selects a specific number (for example, three) ofcandidate scenes 30 sequentially from the highest recommendation level.Then, the image editing apparatus displays in the next-scene displayarea 3 the candidate scenes 30 selected for each of the candidate scenes20A, 20B and 20C.

In the example shown, three candidate scenes, 30D, 30E and 30F, arerecommended as the candidate for the scene (n+1-th scene) following thecandidate scene 20B (n-th scene), and among these, the recommendationlevel for the candidate scene 30E arranged in the middle is the highest.That is, in the example shown, it can be seen that, at the time point ofselecting the n-th scene of the edited work in the scene selection area2, the candidate scenes for the n-th and the n+1-th scenes with thehighest recommendation levels are the candidate scene 20B and thecandidate scene 30E.

As described above, the recommended candidate scenes 30A to 30I that maybe suitably linked after the preceding candidate scenes 20A to 20C arepresented in the next-scene display area 3 on the edit screen 1 for thecandidate scenes 20A to 20C for the preceding scene (n-th scene). Thisenables a user to select any of the candidate scenes 20A to 20C that isto the user's liking in the scene selection area 2 while referring tothe candidate scenes 30A to 30I in the lower level that are presentedfor the candidate scenes 20A to 20C in the upper level and whileconsidering the correlation between scenes of the edited work.

Additionally, in the example of the edit screen 1 of FIG. 1, thecandidate scene for the n-th scene that is currently selected and thecandidate scene 30 for the next n+1-th scene are presented in a treestructure with two layers, but it is not limited to such example, andthe candidate scenes may be presented in a tree structure with three ormore layers, for example.

Next, the preview area 4 will be described. The preview area 4 is anarea for previewing a connecting part of the candidate scene that isselected in the scene selection area 2 (n-th scene of the edited work)and the candidate scenes before and after the selected scene (includingat least the n−1-th and n+1-th scenes). More specifically, the previewarea 4 displays a playback moving image 41 of the connecting part of thecandidate scene 20B (n-th scene) that is selected in the scene selectionarea 2, the candidate scene (not shown) for the preceding scene (n−1-th,n−2-th, . . . ), and the candidate scene 30E for the following scene(n+1-th, n+2-th, . . . ).

The candidate scene 20B here that is selected in the scene selectionarea 2 is the candidate scene that is currently selected by the user(that is, the candidate scene specified by the cursor 21) among theplurality of candidate scenes 20A to 20C that are displayed in the sceneselection area 2. Furthermore, the candidate scene of the scenepreceding the candidate scene 20B (n-th scene) that is selected is oneor more candidate scenes that arc arranged before the candidate scenefor the n-th scene in the edited work, and includes at least thecandidate scene (not shown) for the n−1-th scene. In case the timelength of the n−1-th candidate scene is shorter than a specific time,the candidate scene for the preceding scene may further includecandidate scenes for n−2-th, n−3-th, . . . scenes. Accordingly, a movingimage of a scene of a specific time before the candidate scene for then-th scene will be displayed in the preview area 4. Furthermore, thecandidate scene following the candidate scene 20B (n-th scene) that isselected is one or more candidate scenes arranged after the candidatescene for the n-th scene, and includes at least any of the candidatescenes 30A to 30I for the n+1-th scene. In case the time length of then+1-th candidate scene is shorter than a specific time, the candidatescene for the following scene may further include candidate scenes forn+2-th, n+3-th, . . . scenes. Accordingly, a moving image of a scene ofa specific time after the candidate scene for the n-th scene will alsobe displayed in the preview area 4. In the example shown, the followingcandidate scene is the candidate scene 30E for which the recommendationlevel as the next scene following the candidate scene 20B is the highestamong the candidate scenes 30A to 30I for the n+1-th scene. At the timepoint of selection for the n-th scene in the scene selection area 2, thepreceding n−1-th scene is decided but the following n+1-th scene is notyet decided. After selecting the n-th scene, the n+1-th scene isselected from the candidate scenes 30A to 30I in the next-scene displayarea 3.

Furthermore, the connecting part mentioned above may be the whole of thecandidate scene 20B (n-th) that is selected, the preceding candidatescene (n−1-th, n−2-th, . . . ), and the following candidate scene(n+1-th, n+2-th, . . . ), or it may be parts of these candidate scenes(for example, only the joining portions of these candidate scenes andsurroundings thereof). Additionally, the scene to be displayed in thepreview area 4 may be only the candidate scene 20B that is selected andthe preceding candidate scene, or may be only the candidate scene 20Bthat is selected and the following candidate scene. Furthermore, duringselection of the first scene (n=1) of the edited work, only thecandidate scenes for the first and second scenes are displayed in thepreview area 4. In contrast, during selection of the candidate scene(n=m) for the last scene of the edited work, only the candidate scenesfor the n−1-th and m-th scenes are displayed in the preview area 4.

With the preview area 4 as described above, the playback moving image 41which is the connecting part, played back at high speed (for example,10×), of the candidate scene 20B that is selected in the scene selectionarea 2 and the candidate scenes for the preceding and following scenesis repeatedly displayed (high-speed preview). Furthermore, when thecandidate scene 20B that is selected is changed to the candidate scene20A in the scene selection area 2, the candidate scene 20A after thechange, the candidate scene for the preceding scene and the candidatescene 30B for the following scene are previewed in the preview area 4 athigh speed.

By such preview area 4, the relationship (continuity) between the scenethat a user is paying attention to and the preceding and followingscenes can be presented by displaying the playback moving image 41,which is the scene that is currently selected and an image over aspecific range (for example, several minutes) before and after the scenethat are played back at high speed. Accordingly, the user can repeatedlycheck the continuity between the scene that is selected and thepreceding and following scenes by the playback moving image 41, and thusthe state of correlation between the scene that is selected and thepreceding and following scenes and the acceptability of the overallstructure can be promptly and appropriately checked. As described, for auser to immediately grasp the contexts before and after the scene thatis selected, it is best that the playback moving image 41 is previewedat high speed in the preview area 4. However, the playback speed of thepreview area 4 may be any playback speed as long as no inconvenience iscaused for a user to grasp the playback moving image 41. For example, ifthe playback image is short in its entirety, the playback speed of thepreview area 4 may be a normal playback speed or a slow playback speed.From this viewpoint, although the image editing apparatus automaticallyplays back the image displayed in the preview area 4 at an optimumplayback speed, a user is also enabled to select a desired playbackspeed. Also, the playback speed of the preview area 4 is not limited tobe a constant playback speed, and it may also be a variable speed. Forexample, the image editing apparatus may fast forward a boring scene inthe playback moving image 41.

Next, the roll film area 5 will be described. The roll film area 5 is anarea for displaying, in a roll film format, thumbnail images 50A to 50Eof the first to the m-th scenes making up the edited work that isautomatically edited. The roll film format is a format ofchronologically displaying side by side the thumbnail image (stillimage) of each section of a moving image. The thumbnails of m scenesmaking up the edited work whose recommendation levels by the automaticediting are the highest are displayed chronologically in this roll filmarea 5. The “m” here indicates the total number of scenes making up theedited work, and m is equal to or larger than n.

In the example of FIG. 1, the edited work is made up from five scenes(m=5). The thumbnail 50A on the left end is the thumbnail image of thefirst scene (n=1) of the edited work, and the thumbnail 50E on the rightend is the thumbnail image of the last scene (n=m) of the edited work.Furthermore, a cursor (indicator) 51 is attached to the thumbnail 50C,which is the third from the left in the roll film area 5. This cursor 51indicates the position of the n-th scene (the third scene from thebeginning) that is selected in the scene selection area 2.

Furthermore, when a user moves the cursor 21 in the scene selection area2 and changes the selection of the candidate scene 20, the thumbnail 50of the later candidate scene in the roll film area 5 is also dynamicallychanged according to the changed selection of the candidate scene 20.That is, when the candidate scene 20 for the n-th scene that is selectedin the scene selection area 2 (i.e. that is specified by the cursor 21)is changed from 20B to 20A, the recommended candidate scenes for then+1-th and later scenes (n+1-th to m-th scenes) are also changed.Accordingly, in case there is a selection change for the candidate scene20 in the scene selection area 2, the thumbnails 50C to 50E of the n-thand later scenes (n-th to m-th scenes) are dynamically changed, amongthe thumbnail images 50A to 50E displayed in the roll film area 5. Inthis case, the thumbnail images 50C to 50E are dynamically changed tothe thumbnail image of the candidate scene 20A for the n-th scene afterthe selection change and the thumbnail images of the candidate scenes30B of the n+1-th and later scenes corresponding to the candidate scene20A.

With the roll film area 5 as described above, the overall picture of theedited work that is being edited or the outline of each scene of theedited work can be presented to a user in an easily understandablemanner. Furthermore, in accordance with the selection change for thecandidate scene 20 in the scene selection area 2, the thumbnail images50 of the candidate scene and the later scenes in the roll film area 5are also dynamically changed. Accordingly, a user can select a candidatescene that is to the user's liking in the scene selection area 2 whilereferring to the overall picture of the edited work displayed in theroll film area 5.

Heretofore, the edit screen 1 of the present exemplary embodiment hasbeen described with reference to FIG. 1. According to the edit screen 1of the present exemplary embodiment, a plurality of candidate scenes 20that may be suitably linked after a preceding scene are automaticallypresented in the scene selection area 2 as the recommended images foreach scene of the edited work. Accordingly, a user can edit an imagematerial including a moving image or a still image by simply selecting,sequentially from the first scene of the edited work, a desired scenefrom a plurality of candidate scenes 20 presented in the scene selectionarea 2. Thus, a user can easily produce an image work that is to theuser's liking by an extremely simple operation.

Furthermore, not only the scene selection area 2 but also the next-scenedisplay area 3, the preview area 4 and the roll film area 5 aredisplayed on the edit screen 1. Accordingly, a user can appropriatelyand efficiently produce an edited work that is to the user's likingwhile referring to the images in these areas.

Particularly, the thumbnails 50 of the edit result that is mostrecommended by the automatic editing are shown side by side in the rollfilm area 5. Also, the images shown side by side along the horizontalline in the middle of the edit screen 1 (the candidate scene 20B and thecandidate scene 30E in FIG. 1) also reflect the edit result that is mostrecommended by the automatic editing. By presenting the most recommendededit result on the edit screen 1 in this manner, even the sceneselection becomes unnecessary as long as the user is satisfied with theedit result. Furthermore, when the user selects a candidate scene 20,the candidate scenes in the lower levels are also dynamically changed inthe roll film area 5 and the next-scene display area 3. Accordingly, theuser can easily grasp the relationship between the scene that iscurrently selected and the preceding and following scenes, and thus canselect an appropriate candidate scene with clear determination criteriain mind.

<3. Hardware Configuration of Image Editing Apparatus>

Next, the hardware configuration of an image editing apparatus 100according to the present exemplary embodiment will be described withreference to FIG. 2. FIG. 2 is a block diagram showing the hardwareconfiguration of the image editing apparatus 100 according to thepresent exemplary embodiment.

As shown in FIG. 2, the image editing apparatus 100 includes a CPU 201,a ROM 202, a RAM 203, a host bus 204, a bridge 205, an external bus 206,and an interface 207, for example. Also, the image editing apparatus 100includes an input device 208, a display device 209, a storage device(HDD) 210, a drive 211, a connection port 212, and a communicationdevice 213. As described, the image editing apparatus 100 is configuredby using a general-purpose computer device (for example, a PC), forexample.

The CPU 201 functions as an arithmetic processing apparatus and acontrol apparatus, operates according to various programs, and controlseach unit in the image editing apparatus 100. The CPU 201 performsvarious kinds of processing in accordance with a program stored in theROM 202 or a program loaded from the storage device 210 to the RAM 203.The ROM 202 stores the program and arithmetic parameters used by the CPU201 and functions as a buffer to reduce accesses to the storage device210 from the CPU 201. The RAM 203 temporarily stores the program used bythe CPU 201 when it performs the processing and the parameters and thelike that change as appropriate during the performance. These areconnected to each other by the host bus 204 configured from a CPU bus orthe like. The host bus 204 is connected to the external bus 206 such asa peripheral component interconnect/interface (PCI) bus via the bridge205.

The input device 208 includes, for example, operation means such as amouse, a keyboard, a touch panel, a button, a switch, or a lever, and aninput control circuit that generates an input signal and outputs it tothe CPU 201. The display device 209 is configured from, for example, aliquid crystal display (LCD), an organic EL display, a plasma display, acathode ray tube (CRT) display, or the like. Furthermore, it is equippedalso with a sound output device (not shown) such as a speaker.

The storage device 210 is used to store various kinds of data and isconfigured from, for example, an external or a built-in disk drive suchas a HDD. The storage device 210 drives a hard disk, which is arecording medium, and stores the program executed by the CPU 201 andvarious kinds of data. The drive 211 is used to carry out read and writeoperations to the recording medium and is built in or externallyattached to the image editing apparatus 100. The drive 211 performswrite/read operations of various kinds of data to a removable recordingmedium such as, for example, a magnetic disc, an optical disc, amagneto-optical disc, or a semiconductor memory loaded into the imageediting apparatus 100.

The connection port 212 is used to connect an external peripheral deviceand has a connection terminal such as, for example, a USB or anIEEE1394. The connection port 212 is connected to the CPU 201 or thelike via the interface 207, the external bus 206, the bridge 205, thehost bus 204 and the like. The communication device 213 is acommunication interface configured from, for example, a communicationdevice or the like for connecting to a network 214. The communicationdevice 213 transmits and receives various kinds of data to and from anexternal device having network communication functions, via the network214.

For example, the image editing apparatus 100 can acquire an imagematerial (a moving image or a still image) from an imaging apparatushaving network communication functions or other devices via the network214. The image editing apparatus 100 can also acquire the image materialvia the recording medium in which the image material is recorded by animaging apparatus.

<4. Functional Configuration of Image Editing Apparatus>

Next, the functional configuration of the image editing apparatus 100according to the present exemplary embodiment will be described withreference to FIG. 3. FIG. 3 is a block diagram showing the functionalconfiguration of the image editing apparatus 100 according to thepresent exemplary embodiment.

As shown in FIG. 3, the image editing apparatus 100 includes a materialpreparation unit 110, an analysis unit 120, a scene dividing unit 130, acandidate scene selection unit 140, a switching effect selection unit150, an edit screen creation unit 160, a display control unit 170, adisplay unit 172, an edit result production unit 180, and an input unit182. Also, the image editing apparatus 100 includes a storage unit (notshown), and the storage unit stores a material group database 300, ananalysis result database 310, a user preference database 230, and anedit result database 330. Among the structural elements mentioned above,the material preparation unit 110, the analysis unit 120, the scenedividing unit 130, the candidate scene selection unit 140, the switchingeffect selection unit 150, the display control unit 170, and the editresult production unit 180 are realized by application softwareinstalled in the image editing apparatus 100, for example. That is, theCPU 201 shown in FIG. 2 executes the functions of these units byoperating according to programs stored in the ROM 202 and the like.Additionally, the programs are provided to the image editing apparatus100 via a recording medium or a network.

The material preparation unit 110 prepares an edit material such as amoving image, a still image, sound or the like, that is to be the edittarget of the image editing apparatus 100. The edit material includes,other than an image material such as a moving image or a still image,materials such as audio data, text data and the like. The materialpreparation unit 110 includes an acquisition unit 112, a modificationunit 114, and a generation unit 116.

The acquisition unit 112 acquires the edit material from an externalapparatus. For example, the acquisition unit 112 receives, via anetwork, an image material from an imaging apparatus that captured theimage of the image material. Or, the acquisition unit 112 reads out animage material from a removable recording medium (for example, anoptical disk, a memory card, or the like) in which the image materialwas recorded by the imaging apparatus.

The modification unit 114 generates a material (modified material) whichis an edit material whose quality is improved, by modifying an editmaterial of poor quality. The generation unit 116 generates a newmaterial (generated material) by processing an edit material. Themodified material and the generated material will be described later.

The analysis unit 120 obtains the feature of the edit material(including the feature of the image material) by analyzing the editmaterial stored in the material group database 300, and stores thefeature in the analysis result database 310. For example, the analysisunit 120 detects the feature of the image material by analyzing a movingimage, a still image or audio data included in the image material or byanalyzing information (camera setting value, image capturing time, dataamount, or the like) attached to the image material. The feature of theimage material is information indicating an attribute of the imagematerial, and is information of various kinds of parameters that arequantified to indicate the feature of the image material. The feature ofthe image material will be a reference that is used when dividing amoving image in the image material into a plurality of scenes or whenselecting an optimum candidate scene by the automatic editing.

FIG. 4 shows examples of the features of the image material according tothe present exemplary embodiment. The features shown in FIG. 4 areobtained by analyzing a moving image captured by a camcorder by oneimage capturing operation.

As shown in FIG. 4, the features of the image material may relate to theoperation of the camera (zooming, panning, tilting, hand blur, or thelike), a subject appearing in the moving image, an image which wasunsuccessfully captured, sound, and the like, for example.

The feature relating to a subject appearing in the moving imageindicates the presence/absence of the subject, the movement of thesubject, the presence/absence of a part (for example, the face, thebody) of the subject (person), the expression (a smile) of the subject,or the like. The face or the smile of the subject can be detected byusing a known face detection technology or a known smile detectiontechnology. Also, the presence/absence or the movement of the subjectcan be detected by a focused subject detection and tracking technologybased on the Visual Attention technology. This enables the detection ofa highlight scene in which a focused subject is not making any movement.

Also, the feature relating to a failed image indicates that imagecapturing was unsuccessful due to an image being out of focus, due to ahand blur, or the like, and that the image quality is poor. The failedimage can be detected by a failed scene detection technology that usesblur/out-of-focusness of an image based on the Wavelur technology.Furthermore, the feature relating to sound indicates thepresence/absence or the tempo of the sound (speaking voice,environmental sound, BGM such as music or the like) recorded duringcapturing of a moving image.

As shown in FIG. 4, by analyzing an image material, various kinds offeatures relating to the image material can be obtained, and the changesin these features along the time axis can be obtained. Such temporalchange in the features indicates transition between scenes in a movingimage. Accordingly, one moving image can be divided into a plurality ofscenes based on the various kinds of features.

Accordingly, the scene dividing unit 130 shown in FIG. 3 divides animage material into a plurality of scenes based on the feature of theimage material stored in the analysis result database 310. The scenedividing unit 130 detects dividing positions of a plurality of scenesmaking up one moving image by comprehensively assessing the variouskinds of features in combination with each other according to thepurpose of the editing, and divides the moving image into a plurality ofscenes. Furthermore, the scene dividing unit 130 evaluates whether eachscene is acceptable or not based on the feature of the scene that isobtained by dividing the moving image.

In the example of FIG. 4, one moving image is divided into five scenes Ato E. Scene A is a stable, good scene where, although there is noappearance of the subject, there is no failed image nor hand blur (Goodscene). In contrast, scene B is a scene captured while searching for thesubject with a camera, and there is a hand blur because the camera isbeing moved, and also the subject is moving, and thus the face of thesubject is not properly captured, and a part of the scene is decided tobe a failed image (Bad scene). Furthermore, scene C is the best scenethat includes no blur due to the movement of the camera, includes asmile of the subject, and in which a speaking voice of the subject isrecorded (Best scene). Also, scene D is not as good as scene C in thatthe subject is moving and there is no speaking voice, but otherwise itis a good scene (Good scene). In contrast, scene E is the worst scenefor which there is much hand blur and in which the subject does not makeappearance (Worst scene).

The candidate scene selection unit 140 selects, based on the features ofthe image material, a plurality of candidate scenes that are to be thecandidates for the n-th scene in the edited work among the scenesdivided by the scene dividing unit 130. The candidate scene is thecandidate scene 20 for the n-th scene that is to be presented in thescene selection area 2 on the edit screen 1 described above, and is arecommended scene that may be suitably linked after the preceding n−1-thscene. The candidate scene selection unit 140 selects a specific numberof candidate scenes as the candidates for the n-th scene among aplurality of scenes of the image material by using feature stored in theanalysis result database 310 and user preference information stored inthe user preference database 320 described later. The candidate sceneselection unit 140 sequentially selects a plurality of candidate scenesthat are to be the candidates for the first to m-th scenes of the editedwork starting from the first scene, in accordance with the useroperation on the edit screen 1.

As an example of the candidate scene selection method, a method ofselecting the candidate scenes for the n-th scene based on therecommendation levels calculated from the features will be described. Asthe method of selecting the candidate scenes for the n-th scene for thescene selection area 2, a method that uses the recommendation level foreach scene and a method that uses the recommendation level for theentire automatic editing result can be conceived, for example.

First, the method that uses the recommendation level 340 for each scenewill be described. The candidate scene selection unit 140 obtains therecommendation level for a scene as the candidate for the n-th scenebased on the feature of the scene of the image material and based on therelationship between the feature of the scene and the feature of then−1-th scene. Then, the candidate scene selection unit 140 selects, asthe candidate scenes for the n-th scene, a specific number of scenes forwhich the recommendation levels are high from among the plurality ofscenes of the image material. The recommendation level is a parameterindicating the degree of recommendation for the candidate scene that maybe suitably linked after the preceding scene (n−1-th scene), and iscalculated based on the importance of the scene or the relationship tothe preceding scene. The calculation method of the recommendation levelwill be described later in detail.

By quantifying and obtaining the recommendation level for each scene andcomparing the recommendation levels in this manner, the candidate scenesfor the n-th scene that may be suitably linked after the n−1-th can beappropriately selected.

Next, the method that uses the recommendation level for the entireautomatic editing result will be described. According to this method, aplurality of patterns of automatic editing result are produced, therecommendation level for each automatic editing result is obtained, andthe candidate scenes for the n-th scene are selected.

More particularly, the candidate scene selection unit 140 first createsa plurality of patterns of automatic editing result by combining atleast a part of the scenes among a plurality of scenes obtained bydividing an image material in an arbitrary order. For example, whenthere are scenes A, B, and C, 15 patterns of automatic editing resultwill be created (A, B, C, A+B, B+A, A+C, C+A, B+C, C+B, A+B+C, A+C+B,B+A+C, B+C+A, C+A+B, C+B+A). The automatic editing result is an editedwork that is automatically created by the automatic editing function ofthe image editing apparatus 100. Additionally, the automatic editingresult may be a plurality of scenes arranged in a chronological order orit may be a plurality of scenes shuffled in a random manner regardlessof the chronological order.

Next, the candidate scene selection unit 140 obtains, based on thefeature of a scene that is obtained by dividing the image material andbased on the relationship between the feature of the scene and thefeature of the preceding scene, each of the recommendation levels forthe plurality of patterns of automatic editing result. Therecommendation level here is a parameter indicating the degree ofrecommendation for the automatic editing result, and becomes high or lowdepending on the feature of each scene or the relationship between thefeatures of the scenes before and after. For example, even in a case ofproducing automatic editing results by combining the same five scenes Ato E, the recommendation levels for the automatic editing results willbe different depending on the order of arrangement of the scenes. Theautomatic editing result whose recommendation level is the highest amongthe plurality of patterns of automatic editing result is presented onthe initial edit screen 1 as the most recommended automatic editingresult.

Then, the candidate scene selection unit 140 selects, based on therecommendation level for each of the automatic editing results describedabove, a plurality of candidate scenes for the n-th scene among aplurality of scenes of the image material. For example, the candidatescene selection unit 140 extracts a specific number of automatic editingresults in the descending order of recommendation level among aplurality of patterns of automatic editing results, and selects therespective n-th scenes in the specific number of automatic editingresults and make them the candidate scenes. As described, by selectingthe candidate scene according to the recommendation level, anappropriate candidate scene can be presented in consideration of thefeatures of the scenes of the entire edit result or the relationshipbetween the scenes.

Furthermore, the candidate scene selection unit 140 may also select thecandidate scene based not only on the feature of each scene describedabove but also on the user preference information (informationindicating the result of past selection by a user) stored in the userpreference database 320. The user preference information is updatedbased on the result of past selection by a user. Every time a candidatescene is selected by a user on the edit screen 1, the edit resultproduction unit 180 described later updates the user preferenceinformation based on the selection result. The candidate scene selectionunit 140 changes the weight on the recommendation level or on thefeature for selecting a candidate scene, according to the userpreference information. For example, in case a user frequently selects ascene with a smiling face, the recommendation level described above iscalculated with the weight for the feature indicating a smiling facemade large. As described, by learning the preference of a user in realtime every time a user selects a candidate scene, the preference can beimmediately reflected on the selection of the later candidate scenes.

Furthermore, the candidate scene selection unit 140 selects, for each ofthe selected candidate scenes for the n-th scene, a plurality ofcandidate scenes that are to be the candidates for the n+1-th scene,based on the feature or the relationship of each scenes. The pluralityof candidate scenes for the n+1-th scene are displayed in the next-scenedisplay area 3 on the edit screen 1. The selection process for thecandidate scene for the n+1-th scene is the same as for the candidatescene for the n-th scene described above.

As described above, the candidate scene selection unit 140 sequentiallyselects candidate scenes for the first, second, . . . , m-th scenes ofthe edited work. When the n-th scene is decided by a user in the sceneselection area 2 on the edit screen 1, the candidate scene selectionunit 140 selects, as the candidates for the n+1-th scene, candidatescenes that may be suitably linked after the n-th scene.

Furthermore, the candidate scene selection unit 140 selects, based onthe feature of each scene of the image material, the first to m-thcandidate scenes to be displayed in the roll film area 5 on the editscreen 1. The first to m-th candidate scenes are the most recommendedautomatic editing result by the automatic editing function. For example,as the method of selecting the first to m-th candidate scenes for theroll film area 5, the method that uses the recommendation level for eachscene and the method that uses the recommendation level for the entireautomatic editing result can be conceived as in the case describedabove.

First, the method that uses the recommendation level for each scene willbe described. As in the case described above, the candidate sceneselection unit 140 repeatedly performs the process of obtaining therecommendation level for each scene as the candidate for the n-th sceneon the first to m-th scenes of the edit result. Accordingly, therecommendation levels for the respective scenes as the first to m-thscenes are obtained. Then, the candidate scene selection unit 140selects, as the candidate scenes for the first to m-th scenes,respective scenes for which the recommendation levels are the highest asthe candidates for the first to m-th scenes among a plurality of scenesof the image material. By combining these candidate scenes for the firstto m-th scenes that were selected in this manner, the most recommendedautomatic editing result can be produced and its thumbnail images 50 canbe displayed in the roll film area 5.

Next, the method that uses the recommendation level for the entireautomatic editing result will be described. The candidate sceneselection unit 140 performs the automatic editing process as in the casedescribed above, creates a plurality of patterns of automatic editingresult, and obtains the recommendation level for each of the automaticediting results. Then, the candidate scene selection unit 140 selects,among a plurality of scenes of an image material, the candidate scenesfor the first to m-th scenes that make up the most recommended automaticediting result. The thumbnail images 50 of such candidate scenes for thefirst to m-th scenes are displayed in the roll film area 5 on the editscreen 1.

Furthermore, when the selection for the candidate scene 20 for the n-thscene is changed (the cursor 21 is moved) in the scene selection area 2on the edit screen 1, the candidate scene selection unit 140 reselects,based on the recommendation level described above, each of the candidatescenes for the n+1-th to m-th scenes to be arranged after the candidatescene 20 for the n-th scene after the selection change. Moreparticularly, the candidate scene selection unit 140 selects, as thecandidate for the n+1-th scene that is to be linked after the candidatescene 20 for the n-th scene after the change, a candidate scene with thehighest recommendation level, and selects, as the candidate for then+2-th scene that is to be linked next, a candidate scene with thehighest recommendation level. By repeating such process, the candidatescenes for the n+1-th to m-th scenes corresponding to the candidatescene 20 for the n-th scene after the selection change are respectivelyreselected. This enables to dynamically change the display of thethumbnail images 50 in the roll film area 5 in accordance with theselection change for the candidate scene 20 in the scene selection area2.

As described above, the candidate scene selection unit 140 has theautomatic editing function for an image material. Here, the candidatescene selection unit 140 selects a plurality of candidate scenes as thecandidates for each scene of the edit result. The candidate scenesselected by the candidate scene selection unit 140 are displayed in thescene selection area 2, the next-scene display area 3, the preview area4, and the roll film area 5 on the edit screen 1 as appropriate.

The switching effect selection unit 150 selects, based on the featuresof the candidate scenes selected by the candidate scene selection unit140, candidates for the image switching effect to be applied between thecandidate scenes among a plurality of image switching effects that areset in advance. The image switching effect is also referred to as aneffect, and is an image effect to be applied on the connecting portionof two moving images, and it may be “Cut,” “FadeIn (B),” “Still,”“Wipe,” “Dissolve,” “Insertion (W),” or the like, for example. Byselecting, by the switching effect selection unit 150, appropriatecandidates for the image switching effect to be applied betweencandidate scenes in accordance with the features of the candidatescenes, the candidates for the image switching effect can be displayedby a selection menu 27 (refer to FIG. 11) on the edit screen 1 describedlater. This enables a user to select a desired image switching effectamong appropriate candidates for the image switching effect from theselection menu 27 on the edit screen 1.

The edit screen creation unit 160 reads out the image data of thecandidate scenes selected by the candidate scene selection unit 140described above from the material group database 300, and creates theedit screen 1 as shown in FIG. 1.

The display control unit 170 controls the display unit 172, and makesthe display unit 172 display the edit screen 1 created by the editscreen creation unit 160. The display unit 172 is a monitor device suchas an LCD or the like, and displays the edit screen 1 under the controlof the display control unit 170.

The edit result production unit 180 adjusts, based on the user operationon the input unit 182, the edit result that has been automaticallyedited by the candidate scene selection unit 140 described above, andproduces a final edit result (edited work). That is, the edit resultproduction unit 180 produces the final edit result (edited work) bycombining the candidate scenes for the first to m-th scenes that aresequentially selected by a user among a plurality of candidate scenesthat are displayed in the scene selection area 2 on the edit screen 1.

As shown in FIG. 1, the candidate scenes 20 for the n-th scene of theedit result are displayed in the scene selection area 2 on the editscreen 1. When a user selects a desired scene from the candidate scenesin the scene selection area 2, the candidate scenes for the n+1-th scenewill be displayed in the scene selection area 2 next. By repeating suchuser selection, m scenes (first, second, . . . , m-th scenes) that makeup the edit result are decided sequentially from the beginning.

The edit result production unit 180 connects the m scenes that areselected by the user in the manner described above in the order ofselection, produces an edit result (edited work) that is obtained byediting an image material, and stores information relating to the editresult in the edit result database 330. The information to be storedrelating to the edit result may be the image data itself of the editresult or may be identification information of a plurality of scenesthat make up the edit result.

Furthermore, the edit result production unit 180 determines one imageswitching effect that is selected by the user as the image switchingeffect to be applied between scenes, among a plurality of candidates forthe image switching effect that are displayed in the selection menu 27(refer to FIG. 11) on the edit screen 1. The plurality of candidates forthe image switching effect in the selection menu 27 are the imageswitching effects that may be suitably applied between the scenes, theplurality of candidates being selected by the switching effect selectionunit 150. Such image switching effects are also sequentially determinedfrom the connection of the first two scenes of the edit result by thesame process as for the candidate scenes described above. The editresult production unit 180 stores the information relating to the imageswitching effect to be applied between scenes that is determined in thismanner in the edit result database 330.

Additionally, the input unit 182 may be a touch panel that isoverlappingly arranged on the display unit 172 that displays the editscreen 1, or it may be a mouse, a keyboard, or the like. In case theinput unit 182 is a touch panel, a user selects a desired candidatescene or a desired image switching effect by tapping on the edit screen1. Also, in case the input unit 182 is a mouse, a user moves a mousepointer that is displayed on the edit screen 1 and selects a desiredcandidate scene or a desired image switching effect.

<5. Image Editing Method>

Next, an image editing method that uses the image editing apparatus 100according to the present exemplary embodiment will be described withreference to FIG. 5. FIG. 5 is a flow chart showing an image editingmethod that uses the image editing apparatus 100 according to thepresent exemplary embodiment.

As shown in FIG. 5, the image editing apparatus 100 first imports anedit material group, such as moving images, still images, sound, or thelike, into the apparatus (S100). The image editing apparatus 100acquires an edit material from an external device via the network 214 ora removable recording medium, and stores the same in the material groupDB 300.

Next, the image editing apparatus 100 analyses the image material group,and stores the analysis result in the analysis result DB 310 (S110).Here, the analysis process of the edit material group (S110) will bedescribed in detail with reference to FIG. 6. FIG. 6 is a conceptualdiagram showing an example of the analysis process of the edit materialgroup according to the present exemplary embodiment.

As shown in FIG. 6, the image editing apparatus 100 first analyses amoving image, a still image, sound, or the like, of an edit materialstored in an edit material group DB 302, and stores the analysis resultin an analysis result DB 312 for an edit material group (S111). Theanalysis result of the edit material group may include the features ofthe image material shown in FIG. 4.

Next, the image editing apparatus 100 determines an edit material thatcan be modified based on the analysis result (feature) of the editmaterial group, modifies the image material, and generates a modifiedmaterial (S112). The material modification process includes a hand-blurcompensation process (moving image), a blur compensation process (stillimage), an environmental sound removing process (sound), and the like,for example. The modified material generated by the materialmodification process is stored in a modified material group DB 304.

Furthermore, the image editing apparatus 100 analyses a moving image, astill image, sound, or the like, of the modified material stored in themodified material group DB 304, and stores the analysis result in ananalysis result DB 314 for a modified material group (S114). Theanalysis result of the modified material group also includes, forexample, the features of the image material shown in FIG. 4.

On the other hand, the image editing apparatus 100 newly generates anedit material (hereinafter “generated material”) that is more suitableto be edited from the edit material based on the analysis result of theedit material group (feature) stored in the analysis result DB 312 anduser preference information stored in the user preference DB 320 (S116).The material generation process includes a subject-tracking and croppingprocesses (moving image), a slow motion process (moving image), a stillimage extraction process (moving image), a high definition process(still image), an illustration process (still image), a vocal emphasisprocess (sound), and the like, for example. The generated materialgenerated by the material generation process is stored in a generatedmaterial group DB 306.

Furthermore, the image editing apparatus 100 analyses a moving image, astill image, sound, or the like, of the generated material stored in thegenerated material group DB 306, and stores the analysis result in ananalysis result DB 316 for a generated material group (S118). Theanalysis result of the generated material group also includes, forexample, the features of the image material shown in FIG. 4.

By preparing a material in the manner described above, a modifiedmaterial obtained by modifying an edit material or a generated materialnewly generated from an edit material can also be made an image materialthat is an edit target. Furthermore, not only the raw data of the editmaterial but also the modified material and the generated material areanalysed, and the features of these materials are obtained from theanalysis results. This enables the inclusion of the modified materialand the generated material as the candidate scenes, and thus an editresult with higher quality and that is to the liking of a user can beproduced.

We return to FIG. 5 to continue with the description. After step S110described above, the image editing apparatus 100 selects an imagematerial that is the edit target from the image materials stored in thematerial group DB 300 of the image editing apparatus 100 and selects anedit mode, based on a user input (S120). The edit mode is a form ofautomatic editing using the edit screen 1, and includes a “story-basedmode,” a “music-driven mode,” a “scene drop mode,” and the like, forexample. The story-based mode is an edit mode for arranging scenes of amoving image into a story, making it easy to view. The music-driven modeis an edit mode for selecting scenes in accordance with the BGM such asmusic. The scene drop mode is an edit mode for determining a scene to bedeleted among a plurality of scenes in a moving image. With the imagematerial that is the edit target and the edit mode set by a user, theimage material can be automatically edited by the desired form ofediting. Additionally, this step S120 can be omitted by setting the editmode in advance.

Then, when the user instructs the image editing apparatus 100 to performautomatic editing by, for example, pressing an automatic edit button(S130), the image editing apparatus 100 automatically edits the imagematerial, and selects the candidate scenes (S140). A selection processfor a candidate scene (S140) will be described here in detail withreference to FIG. 7. FIG. 7 is a conceptual diagram showing an exampleof the selection process for a candidate scene according to the presentexemplary embodiment.

As shown in FIG. 7, the image editing apparatus 100 automatically editsvarious kinds of material groups by using the DB group 310, whichincludes analysis result DBs 312, 314 and 316 for various kinds ofmaterial groups and the user preference DB 320, and selects candidatescenes for each scene making up the edit result and the candidates forthe image switching effect to be applied between scenes. The analysisresults of various kinds of material groups include the analysis resultsof the edit material group, the modified material group and thegenerated material group described above. The analysis result DBs 312,314 and 316 are included in the material group DB 300 shown in FIG. 3.

First, the image editing apparatus 100 divides a moving image, among theimage material that is the edit target selected in S120, into aplurality of scenes (S142). More particularly, the image editingapparatus 100 determines a dividing position of scenes included in themoving image based on the values of various kinds of features includedin the analysis results and the changes in these features along the timeaxis, and divides the moving image into a plurality of scenes. Thefeature, for example, is the operation of the camera (zooming, panning,tilting, or the like), the presence or absence of the subject in themoving image, the movement of the subject, the presence/absence of apart (for example, the face, the body) of the subject (person), theexpression (a smile) of the subject or words spoken by the subject, thepresence or absence or the tempo of sound (music, environmental sound,speaking voice, or the like), the presence or absence of a failed imagedue to a hand blur or out-of-focusness, or the like, shown in FIG. 4.

Next, the image editing apparatus 100 obtains the recommendation levelfor an image material based on the feature of an image material, such aseach scene of a moving image obtained in S142, a still image, or thelike (S144). More particularly, the image editing apparatus 100 firstselects, from a plurality of types of features, a plurality of featuresthat are suitable for the edit mode selected in S120, and weights theselected features according to the user preference information in theuser preference DB 320. The features obtained as a result will be abasis for calculating the recommendation level for each scene in themoving image. Next, the image editing apparatus 100 quantifies therecommendation level for each image material based on the feature ofeach scene of the image material included in the analysis result in theanalysis result DB 310 and based on the relationship between thefeatures of the scenes.

A concrete example of quantification of the recommendation level will bedescribed. The recommendation level for each scene obtained by dividingthe moving image is obtained from the feature of the scene. For example,faces of people appearing in the moving image are detected andidentified, and based on the appearance time of each person, his/herposition in the image, and the like, the importance of each person maybe assessed and the recommendation level may be raised for a scene inwhich a person with high importance appears over a long period of time.Alternatively, the recommendation level may be raised for a scene thatenables to grasp the image capturing situation or a scene that enables apanoramic view of the image capturing environment (for example, a scenewith a signboard, a scene of a distant view).

The recommendation level may also be obtained from the order ofarrangement of each scene of the image material (the relationshipbetween the features of scenes). For example, in case a plurality ofscenes having similar features are successively arranged, therecommendation level may be reduced as the scene being tediouslylengthy. Alternatively, in case scenes are arranged in reversed order ofimage capturing time where the edit mode is the story-based mode, therecommendation level may be reduced.

Next, the image editing apparatus 100 selects a plurality of candidatescenes as the candidates for a scene that is currently selected, basedon the recommendation level for each scene obtained in S144 (S146). Forexample, when selecting the n-th scene of the edit result, a specificnumber (for example, three) of candidate scenes are selected from aplurality of scenes of the moving images or from the still images in thedescending order of recommendation levels for the scenes as the n-thscene. The candidate scene is a scene suitable as the scene after then−1-th scene that is already determined.

Furthermore, the image editing apparatus 100 selects, as the imageswitching effect to be applied between each scene, candidates for animage switching effect appropriate for the feature, based on the featureof each image material described above (S148). For example, when thereis a time gap between two successive scenes, a black image is insertedbetween the scenes, and the image switching effect “FadeIn (B)” thatmakes the scenes fade in/fade out is selected. Furthermore, when thereis a large difference between the features of the scenes, an imageswitching effect that make the scenes partially overlap with each otheris selected. With the image editing apparatus 100 selecting candidatesfor the image switching effect to be applied between each scene, a usercan determine a desired image switching effect from the candidates.Additionally, the image editing apparatus 100 may also automaticallyselect the optimum image switching effect to be applied between eachscene.

According to the candidate scene selection process S140 as describedabove, a plurality of candidate scenes that may be suitably linked aftera preceding scene are automatically extracted from a plurality of imagematerials (scenes of a moving image, still images) and candidates forthe image switching effect to be applied between each scene are alsoselected, based on the features of the image materials.

We return to FIG. 5 to describe the processes following S140. The imageediting apparatus 100 displays the edit screen 1 (refer to FIG. 1) forpresenting, to the user, the plurality of candidate scenes selected inS140 (S150). As shown in FIG. 1, a plurality of candidate scenes 20A to20C that are suitable as the candidate for the n-th scene that isselected are displayed in the scene selection area 2 on the edit screen1. Also, a plurality of candidate scenes 30A to 30C, 30D to 30F, and 30Gto 30I for the n+1-th scene are displayed in the next-scene display area3 respectively for the candidate scenes 20A to 20C. Furthermore, ahigh-speed playback moving image 41 is displayed in the preview area 4for checking the continuity between the scene that is selected and thescenes before and after the scene. Also, the thumbnail images 50A to 50Eshowing the overall picture of the edit result that is most recommendedby the automatic editing are displayed in the roll film area 5.

The user looks at this edit screen 1, grasps the contents of thecandidate scenes and the contexts before and after the scene, andselects a scene that is to the user's liking from the plurality ofcandidate scenes 20A to 20C presented in the scene selection area 2(S160). The user selects the desired scene by moving the cursor 21 upand down by pressing the up/down buttons 22A and 22B adjacent to thecursor 21, setting the cursor 21 to a desired candidate scene 20, andpressing a determination button (not shown). The n-th scene of the editresult is decided by such user selection.

Next, the image editing apparatus 100 updates the user preference DB 320based on the result of the user selection in S160 (S170). This enablesto store the history of the user selection results in the past in theuser preference DB 320. The image editing apparatus 100 updates the userpreference information in the user preference DB 320 according to thefeature of the candidate scene that the user selected in S160. Forexample, when the user selected a scene with a smile, the userpreference information is updated such that the weight on a featurerelating to a smile is large at the time of calculating therecommendation level in S140 described above. This enables to reflectthe user selection results in the past from the next time the candidatescenes are to be selected in S140.

Next, the image editing apparatus 100 decides whether all the scenes(first to m-th scenes) of the edit result are selected by the user ornot (S180). In the case not all the scenes are selected, n isincremented by 1 (S182), and the process by the image editing apparatus100 returns to S140. Then, as in the same manner as above, the imageediting apparatus 100 selects a plurality of candidate scenes for thenext scene (n+1-th scene) (S140), presents the candidate scenes on theedit screen 1 (S150), and determines the candidate scene selected by theuser on the edit screen 1 as the n+1-th scene (S160). By sequentiallyrepeating such selection process for candidate scenes, all the scenes(first to m-th scenes) will be determined sequentially from the firstscene of the edit result.

Then, the image editing apparatus 100 combines the images of the firstto m-th scenes sequentially selected by the user in the manner describedabove, produces a final edit result (edited work), and stores the samein the edit result DB 330 (S190).

<6. Application Example>

Next, an application example of the image editing method that uses theedit screen 1 according to the present exemplary embodiment will bedescribed.

(1) Editing According to Edit Mode

First, the image editing method in accordance with the edit modeaccording to the present exemplary embodiment will be described withreference to FIGS. 8 to 10.

The image editing apparatus 100 according to the present exemplaryembodiment can present various edit modes according to the purpose ofthe user. As described above, as the edit mode, there are thestory-based mode, which is for arranging scenes of a moving image into astory, making it easy to view, the music-driven mode, which is forselecting scenes in accordance with the BGM such as music, the scenedrop mode, which is for determining a scene to be deleted among aplurality of scenes in a moving image, and the like. The user can easilyobtain a desired edit result by setting a desired edit mode according tothe purpose.

FIG. 8 is an example of display of the edit screen 1 in case of editingin the story-based mode. In the story-based mode, an edit result with astory line is obtained by deleting unnecessary scenes from one or moreimage materials and chronologically connecting scenes that arenecessary. The unnecessary scenes are a failed scene with hand blur orwhich is out of focus, a scene in which a desired subject does not makeappearance, an unlively scene, a tediously lengthy scene, and the like.The scenes that are necessary are a scene in which a desired subjectappears, a scene with a smile, a lively scene, a scene with aconversation, a highlight scene, a digest scene, and the like. In thestory-based mode, candidate scenes made up of the scenes that arenecessary are presented in the scene selection area 2 on the edit screen1 basically in a chronological order. Accordingly, a user can easilyobtain a desired edit result with a story line by sequentially selectinga candidate scene presented on the edit screen 1.

FIG. 9 is an example of display of the edit screen 1 in case of editingin the music-driven mode. In the music-driven mode, an edit result isobtained by connecting scenes matching the music played as BGM. Aspecific part in a music piece (for example, a chorus, an A melody, a Bmelody, and the like) can be specified by a known 12 tone analysistechnology, and thus, in the music-driven mode, a scene of an image canbe displayed in association with a part of a music piece. For example, ahighlight scene may be associated with the chorus part of a music piece,and a peaceful scene may be associated with the A melody part. Icons 25and 35 indicating the part of the music piece are displayed on the editscreen 1 of FIG. 9 to present to a user which part of the music piecethe scene that is currently selected is associated with. The icon 25indicates that the n-th scene that is currently selected in the sceneselection area 2 corresponds to the A melody part of the music piece,and the icon 35 indicates that the n+1-th scenes displayed in thenext-scene display area 3 correspond to the chorus part of the musicpiece. This allows the user to select on the edit screen 1 a candidatescene 20 that matches the chorus part or the like of a music piece.

FIG. 10 is an example of display of the edit screen 1 in case of editingin the scene drop mode. In the scene drop mode, an edit resultconfigured from scenes that are necessary can be obtained by deletingunnecessary scene by using a delete icon 26 displayed in the sceneselection area 2 on the edit screen 1. As shown in FIG. 10, delete icons26A to 26C are displayed on respective candidate scenes 20 that aredisplayed in the scene selection area 2 on the edit screen 1. A user candelete the candidate scene 20A corresponding to the delete icon 26A byselecting the delete icon 26A on the edit screen 1. In the case a userselects one candidate scene 20 at the time of selecting the n-th scenewithout selecting any of the delete icons 26, the process proceeds tothe selection of the next n+1-th scene with none of the candidate scenes20A to 20C deleted.

As described above, the preview area 4 on the edit screen 1 displays thehigh-speed playback moving image 41 showing the selected scene and thescenes before and after the selected scene. Accordingly, by deleting anunnecessary scene by using the delete icon 26 on the edit screen 1, thedisplay of the preview area 4 changes, and a user can check theconsequences of deleting the scene on the edit screen 1. There was noGUI in the past that immediately presented to a user the consequences onthe overall edit result caused by the deletion of an image material.However, the edit screen 1 according to the present exemplary embodimentis capable of providing such GUI, and the issue of the past can besolved.

(2) Selection of Image Switching Effect

Next, a method of selecting the image switching effect on the editscreen 1 according to the present exemplary embodiment will be describedwith reference to FIG. 11.

As shown in FIG. 11, when the left button 23A (or the right button 23B)in the scene selection area 2 on the edit screen 1 is pressed, theselection menu 27 for selecting the image switching effect is displayed.The selection menu 27 displays, in the descending order ofrecommendation level, candidates for the image switching effect to beapplied between the candidate scene 20B for the n-th scene that isselected and the candidate scene for the preceding or following scenes(n−1-th or n+1-th). In the example of FIG. 11, six image switchingeffects are presented in the selection menu 27 in the descending orderof recommendation level from the top, and it can be seen that “Cut” isthe most recommended image switching effect. These candidates for theimage switching effect are automatically selected based on the featuresof the candidate scenes, and are appropriate image switching effectsthat are in accordance with the contents or the features of both scenes.

A user can select an image switching effect that is to the user's likingfrom the appropriate candidates for the image switching effect that arein accordance with the contents or the like of the scenes, by moving acursor 28 in the selection menu 27. Also, since the image switchingeffects in the selection menu 27 are presented in the order ofrecommendation level, and it is highly convenient at the time of theuser selecting an image switching effect. The image editing apparatus100 determines the one image switching effect selected by the user asthe image switching effect to be applied between the scenes, and storesthe same in the edit result DB 330.

(3) Insertion of Caption

Next, a method of inserting a caption using the edit screen 1 accordingto the present exemplary embodiment will be described with reference toFIG. 12.

As shown in FIG. 12, in the scene selection area 2 on the edit screen 1,a caption 29 (text information) can be added to the candidate scene 20that is selected. According to the edit screen 1 of the presentexemplary embodiment, the caption 29 can be easily inserted in the firstto m-th scenes making up the edit result in this manner, and the task ofediting can be performed efficiently.

<7. Summary>

Heretofore, the image editing apparatus 100 according to the presentexemplary embodiment and image editing methods that use the imageediting apparatus 100 have been described in detail. According to thepresent exemplary embodiment, a plurality of candidate scenes 20 thatmay be suitably linked after the preceding scene are presented in thescene selection area 2 on the edit screen 1. Accordingly, a user isallowed to easily produce an image work that is to the user's liking bysequentially selecting a desired scene from the plurality of candidatescenes 20 presented on the edit screen 1. Accordingly, a plurality ofrecommended scenes that may be suitably linked after a certain scene canbe presented at the time of adjustment of an image work that has beenautomatically edited, and also, a user is allowed to easily produce animage work that is to the user's liking with a simple operating system.

More particularly, the image editing method that uses the edit screen 1according to the present exemplary embodiment can provide an extremelysimple operating system at the time of manual adjustment of an editcontent after the automatic editing of the moving image. That is, a usercan easily edit a moving image and produce an image work that is to theuser's liking by simply sequentially selecting from the first scene aplurality of candidate scenes presented on the edit screen 1. At thistime, the user can elaborate on the desired content by trial and errorby appropriately selecting and changing, as many times as the userwants, between the plurality of candidate scenes 20 presented on thescene selection area 2.

Furthermore, since the recommendation level for each image material isobtained based on the analysis result of the image material and anoptimum automatic editing result is produced before the manual editing,the image editing apparatus 100 can display the most recommendedautomatic editing result on the edit screen 1. For example, the imagesshown side by side along the horizontal line in the middle of the editscreen 1 (the candidate scenes 20B and 30E in FIG. 1), and thethumbnails 50A to 50E in the roll film area 5 are the most recommendedautomatic editing result according to the automatic editing. With themost recommended edit result presented on the edit screen 1, even theselection of scenes becomes unnecessary as long as the user is satisfiedwith the edit result.

Furthermore, on the edit screen 1, not only is the candidate scene(n-th) that is currently selected in the scene selection area 2displayed, but also the candidate scenes for the next scene (n+1-th) aredisplayed in the next-scene display area 3. Accordingly, the user cancheck both scenes at the same time. In addition, the playback movingimage 41 for the candidate scene that is selected and the candidatescenes for the preceding and following scenes is displayed in thepreview area 4 at all times so as to be previewed at high speed.Accordingly, the user can check in an instant the continuity between thecandidate scene that is selected and the preceding and following scenes.Also, information (the icon 23) that enables to understand the reasonfor recommendation of the candidate scene 20 is also shown in the sceneselection area 2, and such information helps the user to select acandidate scene 20.

Furthermore, every time the user selects a candidate scene 20 or animage switching effect to be applied between scenes in the sceneselection area 2, the image editing apparatus 100 updates the userpreference DB 320 by using the selection result. This enables to learnthe preference of a user in real time and to immediately reflect thepreference on the selection of the later candidate scenes.

Furthermore, it is also possible to generate, as an image material inwhich a plurality of image materials are mixed, an edit result in whichboth a moving image and a still image are mixed. Furthermore, the usercan easily insert a caption (text information) in the edit result on theedit screen 1. Furthermore, the user can check, on the edit screen 1,not only the plurality of candidate scenes that are used in the editresult (edited work) but also scenes that are not used.

Furthermore, the image editing method according to the present exemplaryembodiment is convenient when editing by combining various imagematerials that include a plurality of moving images, still images, orthe like, from different generation sources. Exemplary uses of the imageediting method according to the present exemplary embodiment are asfollows.

For example, a case is assumed where members of a family went to schoolon a field day to cheer for the youngest child and where each membercaptured images of the sports event using an imaging apparatus that wasallotted to the member. At this time, it is assumed that the father useda digital camcorder, the mother used a compact digital camera, the elderbrother used a single-lens reflex digital camera, and the elder sisterused a built-in camera of a mobile phone. Then, it was decided to editthe image materials captured by these imaging apparatuses into one shortmovie (moving image) as a souvenir of the youngest child's energeticperformance on the field day. The task of editing was decided to beperformed by the mother unfamiliar with a personal computer instead ofthe busy father, and the mother decided to use the automatic editingsoftware that could be easily operated by anyone.

Even in such use case, the mother can easily produce one edited work(short movie) in which various image materials are combined by using theimage editing apparatus 100 installed with software with the automaticediting function according to the present exemplary embodiment.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

For example, although an example has been described for the exemplaryembodiment described above where the image editing apparatus is appliedto a personal computer, the present invention is not limited to suchexample. As shown in FIG. 13, the image editing apparatus of the presentinvention can be applied to a video camera 100A, a portable game machine100B, a combination of a game machine 100C and a television 100D, amobile phone 100E, or the like, as long as it is a device capable ofdisplaying the edit screen 1 when the editing application software isinstalled.

What is claimed is:
 1. An information processing apparatus comprising:circuitry configured to determine at least one recommendation level forat least one of the plurality of scenes of an edited work, the at leastone recommendation level being determined in part based on a similarityrelationship between features of the plurality of scenes, toautomatically select, for an n-th scene of an edited work, a pluralityof first candidate scenes from the plurality of scenes based on the atleast one recommendation level, and to generate information related to agraphical user interface including a scene selection area and a previewarea displayed concurrently on a display, wherein the scene selectionarea includes one or more first candidate images corresponding to atleast one of the automatically selected plurality of first candidatescenes, and the preview area includes a video preview of the preceding(n−1-th) scene.
 2. The information processing apparatus according toclaim 1, further comprising: a hardware memory configured to store theplurality of scenes.
 3. The information processing apparatus accordingto claim 2, wherein the preview area includes a video preview of thepreceding (n−1-th) scene and a selected one of the plurality of firstcandidate scenes.
 4. The information processing apparatus according toclaim 2, wherein the preview area includes the preview of the preceding(n−1-th) scene, a selected one of the plurality of first candidatescenes, and a following (n+1-th) scene selected by the circuitry.
 5. Theinformation processing apparatus according to claim 1, wherein theprocessor is configured to determine a different recommendation levelfor each of the plurality of scenes, and to select the plurality offirst candidate scenes from the plurality of scenes based on thedifferent recommendation levels.
 6. The information processing apparatusaccording to claim 5, wherein the recommendation level for each of theplurality of scenes is reduced in response to determining the pluralityof scenes have similar features.
 7. The information processing apparatusaccording to claim 5, wherein the recommendation level for each of theplurality of scenes is reduced in response to determining successivelyarranged scenes of the plurality of scenes have similar features.
 8. Theinformation processing apparatus of claim 5, wherein the recommendationlevel for each of a plurality of scenes of the edited work is furtherdetermined based on an edit mode.
 9. The information processingapparatus according to claim 5, wherein the circuitry is configured togenerate the information relating to the graphical user interface suchthat the plurality of first candidate scenes are displayed in the sceneselection area based on the different recommendation levels of theplurality of first candidate scenes.
 10. The information processingapparatus according to claim 1, wherein the information related to thegraphical user interface includes an indicator, in the scene selectionarea, that identifies a selected one of the plurality of first candidatescenes based on a user input.
 11. The information processing apparatusaccording to claim 1, wherein, based on the plurality of first candidatescenes, the circuitry is configured to select, for a following (n+1-th)scene of the edited work, a plurality of second candidate scenes fromthe plurality of scenes, and the information related to the graphicaluser interface includes a next scene display area, the next scenedisplay area including one or more second candidate images correspondingto the plurality of second candidate scenes.
 12. The informationprocessing apparatus according to claim 1, wherein the circuitry isconfigured to automatically select at least one scene of the edited workfollowing the n-th scene, and the information related to the graphicaluser interface includes a roll film area, the roll film area includingthumbnail images corresponding to at least the n-th scene and theautomatically selected at least one scene included in the edited work.13. The information processing apparatus according to claim 1, whereinthe information related to graphical user interface includes icons, inthe scene selection area, that represent the features of the pluralityof first candidate scenes.
 14. The information processing apparatusaccording to claim 1, wherein the information related to the graphicaluser interface includes an image switching effect selection window, theimage switching effect selection window including a list of userselectable image switching effects to be applied to a connection portionbetween the n-th scene and one of the preceding (n−1-th) or following(n+1-th) scenes of the edited work.
 15. The information processingapparatus according to claim 1, wherein the information related to thegraphical user interface includes a delete icon for each of the one ormore plurality of first candidate scenes, in the scene selection area,and the circuitry is configured to delete a corresponding one of theplurality of first candidate scenes in response to receiving a userselection of the delete icon.
 16. The information processing apparatusaccording to claim 1, wherein the information related to the graphicaluser interface includes a caption entry field corresponding to one ofthe plurality of first candidate scenes.
 17. The information processingapparatus according to claim 1, wherein the circuitry is configured toselect the plurality of first candidate scenes from the plurality ofscenes based on a chronological order of the plurality of scenesoccurring after the preceding (n−1-th) scene of the edited work, and togenerate the information related to the graphical user interface suchthat the one or more first candidate images included in the sceneselection area are displayed according to the chronological order. 18.The information processing apparatus according to claim 1, wherein thecircuitry is configured to select the plurality of first candidatescenes from the plurality of scenes based on a portion of a music pieceto be associated with the n-th scene of the edited work.
 19. A method ofan information processing apparatus for generating an edited workincluding a subset of a plurality of scenes in an image material, themethod comprising: determining at least one recommendation level for atleast one of the plurality of scenes of an edited work, the at least onerecommendation level being determined in part based on a similarityrelationship between features of the plurality of scenes; automaticallyselecting, for an n-th scene of an edited work, a plurality of firstcandidate scenes from the plurality of scenes based on the at least onerecommendation level; and generating, via a processor, informationrelated to a graphical user interface including a scene selection areaand a preview area displayed concurrently on a display, wherein thescene selection area includes one or more first candidate imagescorresponding to at least one of the automatically selected plurality offirst candidate scenes, and the preview area includes a video preview ofthe preceding (n−1-th) scene.
 20. A non-transitory computer-readablestorage medium having computer-readable instructions thereon which whenexecuted by a computer cause the computer to perform a methodcomprising: determining at least one recommendation level for at leastone of the plurality of scenes of an edited work, the at least onerecommendation level being determined in part based on a similarityrelationship between features of the plurality of scenes; automaticallyselecting, for an n-th scene of an edited work, a plurality of firstcandidate scenes from the plurality of scenes based on the at least onerecommendation level; and generating information related to a graphicaluser interface including a scene selection area and a preview areadisplayed concurrently on a display, wherein the scene selection areaincludes one or more first candidate images corresponding to at leastone of the automatically selected plurality of first candidate scenes,and the preview area includes a video preview of the preceding (n−1-th)scene.